Skip to main content

Cloud-Native Storage Orchestrator: Rook

Introduction

Kubernetes* (or K8s)-based cloud-native apps are extensively deployed in production environments, posing a problem: how do you integrate a conventional storage system into a Kubernetes cluster? As a solution, CNCF offer Rook.

Rook is an open source cloud-native storage orchestrator that makes storage software self-managing, self-scaling, and self-healing. Ceph, EdgeFS, Cassandra, CockroachDB*, NFS, and YugabyteDB are among the storage systems supported by Rook.

Operator Pattern

The Rook framework's core is the Rook operator. Operator is a Kubernetes controller that manages apps and their components using CRs (Custom Resource).

The Kubernetes controller keeps an eye on the condition of your cluster resource and attempts to get it closer to the intended state. Each controller is in charge of a certain resource, and the reconciliation loop is used to implement it. The reconciliation is initiated when the monitored resource is created, modified, or destroyed.

Rook for Ceph

The Rook operator is a straightforward container that contains everything required to set up and manage a storage cluster. Ceph daemon pods such as MONs, OSDs, MGR, and others are started and monitored by the operator. It also keeps an eye on the daemon to make sure the cluster is running well. When required, Ceph MONs are begun or failed over. As the cluster expands or decreases, other modifications are performed.

Rook-Ceph provides block, filesystem, and object storage for apps, much like native Ceph.

  • Ceph CSI (Container Storage Interface) is a standard that allows containerized workloads on Container Orchestration Systems like Kubernetes to access arbitrary block and file storage systems. Ceph CSI is a Rook integration that allows for two scenarios:
  • RBD (block storage): This driver is tuned for RWO pod access, which means that only one pod may access the storage at a time.
  • CephFS (filesystem): This driver enables RWX to access the same storage from one or more pods.
  • Rook enables the creation of new buckets and access to existing buckets through two custom resources: Object Bucket Claim (OBC) and Object Bucket Access (OBA) for object storage (OB). RGW allows applications to access the objects.

There are three ways to configure Ceph for Ceph cluster maintainers:

  • Toolbox + Ceph CLI: Using a Rook Toolbox pod to run Ceph commands is recommended.
  • The native Ceph dashboard is the second preferred approach. This approach has the same priority as Ceph CLI configuration.
  • Ceph.conf overrides ConfigMap for advanced settings. Some parameters are difficult to change through CLI or dashboard. Ceph CLI, for example, does not allow us to remove MONs directly. Overriding configMap is the only option.

Programmers can develop their own plugin by using the Kubernetes client API to access ConfigMap or Toolbox pod to manipulate a Rook-Ceph cluster.